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Why SOFIA? 



• SOFIA is the largest portable telescope in the world 

- 2.5-meter (100-inch) telescope in a Boeing 747SP 

• SOFIA is reconfigurable and highly flexible 

- Every SOFIA flight series is comparable to a Hubble 
reservicing mission 

o Science instruments can be routinely changed and upgraded 

- Able to quickly respond to all astronomical events 

• SOFIA will exceed the performance of ground-based 
Infrared Telescopes 

- Flies above 99.9% of the water vapor 

• SOFIA is designed to be productive 

- 140 eight-hour research flights per year; 20 year lifetime 

• SOFIA cost much less than space-based observatories 
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Major Components of SOFIA 



Observatory 



Telescope Assembly 


Science and Mission 
Operations Center 



Aircraft 

Operations Center 
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Background 



• SOFIA was established as a 80/20 partnership 
between the U.S. (NASA) and Germany (DLR) 

- Original NASA/DLR MOU signed 1996 

- Germany supplied telescope assembly and other significant 
contributions 

- NASA supplied modified aircraft and Science Operations 
Center 

- NASA receives 80% of available science time, DLR 20% 

• Initial program model was contractor led with NASA 
oversight (privatized) 

• Overtime, a series of schedule slips, cost increases, 
contract issues and mishaps occurred 
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• NASA withheld funding and the Program was slated 
for cancellation in the spring of 2006 

- Members of Congress, Germans and the Science 
Community “pressured” NASA to continue Program 

- NASA commissioned an independent review team to 
consider options 

• The Agency approved the Program for continued 
funding in the fall 2006 

- The Program was restructured: 

o Government led, contractor supported 
o Program management moved to Dryden 
o Two projects; Science and Platform 
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SOFIA Program Organization 
During Development 
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SOFIA SE&I Approach 


Ron Ray 
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SOFIA Systems Engineering 

Transition 



• The new Program Office initiated an independent 
review of SOFIA Systems Engineering (Summer 2006) 

• The SE&I Lead position was transferred to Dryden 

(Sept. 2006) 

- Reviewed SOFIA SE&I history & existing processes 

- Reviewed the SE&I independent assessment and 
recommendations 

- Completed additional assessments 

• Several significant issues with SE&I were identified 

(See following pages) 
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SOFIA SE&I Assessment 




• System Requirements were lacking and fragmented 

- Needed government ownership and greater priority 

- Only a small percentage of Specifications had been 
baselined 

- The Interface Control Documents (ICDs) were not centrally 
managed (not clear who owned what) 

• Program CM process was dysfunctional (over 100 
documents were tied up in the old process) 

- A small group made all of the decisions creating a bottleneck 

- Hardware was being built to unapproved documents 
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SOFIA SE&I Assessment (Cont.) 



• Program had an immediate need for a formal Risk 
Management Process 

- New PM was working this informally because the previous 
system was unmanageable 

• The amount of information already assembled for 
SOFIA was vast and users had difficultly finding things 
on the central Data Management System 

- Over 100,000 data records existed in hard and soft copy 

- Over 50,000 Telescope Assembly (TA) documents existed in 
the Data center only in hard copy and filed chronologically 

- Many documents were owned and managed by the 
Contractors using various document control processes 
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SOFIA SE Lessons Learned 



It is never too late to fix Systems 
Engineering (SE) deficiencies 
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SOFIA Program Systems 
Engineering Dilemma 



• The new Program Management faced a major 
dilemma with Systems Engineering: 

- Either stop and “fix” the Systems Engineering and Integration 
(SE&I) problems identified at the time of transition 


- Continue at risk and try to “rebuild” SE&I along the way 

• Some consideration factors 

- Priority was to get the aircraft from Waco to Dryden and 
demonstrate progress after the threat of cancellation 

- The near-term challenges were not considered as difficult as 
the long-term challenges 

- The new Dryden team members were still coming up to 
speed on the SOFIA systems 


or 
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New Implementation 
Strategy for SE&I 



• The Program made the decision to continue at risk and 
“rebuild” SE&I as we go 

• Risk mitigation decisions/activities 

- Phase the remaining development into increments which 
would give key SE activities a chance to catch-up 

o Add an “Early Science” Milestone to recapture schedule 

o Conduct both near-term and long-term SE activities 
simultaneously 

- Work more collaboratively between the stakeholders and 
developers to compensate for requirement gaps 

o Conduct a series of “delta” System Requirements Reviews 
focusing on near-term needs and requirements 

o Implement cross-Project Integrated Product Teams (IPTs) 

- Establish a new set of SE&I priorities and provide a dedicated 
staff to facilitate the rebuilding process 
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The Use of Risk Management 

on SOFIA 



• SOFIA distributed Program and Project level risks and made Risk 
Management an integrated but delegated management tool 


• The SOFIA Program focused on the “top priority” risks and tracked 
a larger set of “threats” (potential risks) 




Criticality LxC Trend 
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— r Unchanged 
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Approach 
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W - Watch 
A - Accept 
R - Research 


Rank & 
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Risk Title 
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MCCS Development 

tr 

2 
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Loss of Science Community and 
DLR Support, Due to Late Science 


3 
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M 

Cavity Door System Failure ( Loss 
of Science / TA Damage) 

<=> 
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Lack of Requirements Definition 
(Systems Engineering) 
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Mirror 
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Schedule and Cost Growth, Due to 
Schedule Uncertainty 
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Risk List from March 28, 2008 


SOFIA identified the lack of 
Requirements Definition as a risk 
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SOFIA SE Lessons Learned 




Breaking complex development 
activities into increments can improve 
the overall chance of success 


“Sometimes the questions are complicated and the answers are 
simple.” 

Dr. Seuss 
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New SOFIA Life Cycle: 
Incremental Development 


Status - Sept 2007 
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Advantages of the Incremental 
Development Life Cycle for SOFIA 


• Allowed science data to be obtained significantly sooner helping retain 
science community support 

• Allowed requirements time to catch-up over the long term 

• Allowed integration issues to be identified and better isolated as system 
complexity grew 

• Allowed for Observatory performance to be assessed earlier 

- Early 1 st Light gave initial indication we have no major performance 
deficiencies 
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SOFIA SE Technical Priorities 



• Organize and establish Systems Engineering leads and support 
teams for key SE tasks 

- Established a dedicated Requirements Manager (High Priority) 

• Revise Program SE&I documents and processes 

- Risk Management - IT Management 

- Configuration Management - Data Management 

• Develop a new Systems Engineering Management Plan (SEMP) 
to define technical process and requirements 

- Complies with NPR7123.1 

• Establish Program Management Control Boards 

- PMB: Programmatic Control - OCCB: Observatory Control 

• Establish a SOFIA Observatory-Level IPT (SOLIPT) 

- Addresses Observatory and “cross project” technical issues 

• Establish a process to manage and track the status critical 
Program and technical documents 
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SOFIA Program 
SE&I Organization 



System-Level: 

• Design Solution 

• Implementation 

• Integration 

• Verification 


Configuration 

Management 


SOFIA 

Program 

Manager 


SE&I 

Manager 


Program 

Chief 

Engineer 


Project 

Scientists 


Observatory-Level: Observatory-Level 

• System Integration . Requirements 

• V&V • Validation 

• Discrepancy Resolution • Performance 

• Risk/Threat Mitigation 


Data 

Management 


IT Systems 
Management 


Technical 
Planning & 
Support 


• Specifications 

• ICDs 

• ORDs 

• Database Mgt 

• Req. Traceability 

• V&V Tracking 


Program PMB 
Observatory OCCB 
Platform PCB 
Science PCB 
Contractor CCBs 
CM Records 


• Program DM 

• Platform Proj. DM 

• Science Proj. DM 

• Science Data 

• SOFIA Data Center 

• Export Control 

• Records Retention 

• Data Mgt. System 


Program IT 
Platform Proj. IT • 
Science Proj. IT • 
IT Security 
System Admin. 
Account Mgt. 

Sys. Development 


SE&I Plans 
SE&I Processes 
Technical Reviews 


9 February 2011 


PM Challenge 


22 












SOFIA SE Lessons Learned 




Management needs clear insight on 
the status of SE products 
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SOFIA Program SE&I 
Documentation Tracking Tool 


Status on 12/01/2007 
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• Illustrates a summary chart presented to SOFIA Management to track 
documentation progress 

See conclusion chart for more recent status 


PM Challenge 


9 February 2011 


24 


SOFIA SE Lessons Learned 




SE must account for and tailor to 
various Center and cultural differences 


“Scientists investigate that which already is; engineers create that 
which has never been.” 

Albert Einstein 
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The Relationship Between 
Engineers and Scientists 



• Engineers and Scientists must have clear and distinct roles and 
responsibilities 

• On SOFIA (during the development phase) the Scientist is the 
“customer” and the Engineer is the “implementer” 

- SE&I is often the interface 



Scientists: 

• Specify needs and requirements 

• Develop the Concept of Operations 

• Participate in technical reviews 

• Accept verification 

• Provide validation 


Engineers: 

• Interpret and decompose requirements 

• Conduct trade studies 

• Develop design & implementation strategies 

• Provide verification 

• Participate in validation 


\ 


Systems Engineer: 

• Manages requirements 

• Implements supporting processes 

• Establishes entrance/exit criteria for technical reviews 

• Maintains the V&V Matrix 
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SOFIA SE Lessons Learned 


“Better is the enemy of good 
enough” 


• Engineers want to know what are the minimum requirements 
so they can meet them 

• Scientists want the best they can get with no constraints: 

“Good enough is the enemy of the greaf 
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Systems Engineering is an 
Optimization Process 



• Too little or too much SE causes problems 

- SE must be “value added” 

• When addressing SE&I in the middle of a Program, 
there is never enough time, resources, and budget to 
complete all processes 

- SE priorities must be developed and documented but also 
must fit within the overall Program/Project priorities 

• SOFIA used the Risk Management process to 
understand and accept the risks of “deliberately” 
leaving some things out due to schedule and budget 
realities 
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Requirements Management 


Mike Brignola 
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SOFIA SE Lessons Learned 




Making the “Lack of Requirements 
Definition” a Program risk, is an 
effective way to highlight and address 
the problem 

• This allowed the Program Management to establish a long-term 
mitigation strategy to drive down the risk 

• SOFIA Management made a long-term commitment to correcting 
requirements deficiencies 
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Requirements Deficiency 
Risk Mitigating Act i ons 

1. Develop SE plan and process to audit, develop, and manage 
requirements. Implement early with adequate staff 

• Utilize Product/Specification Tree to facilitate communication 

• Utilize RM Database tool to manage 4000 requirements, trace and allocate 

2. Establishing a NASA Requirements Manager with broad systems 
knowledge to bridge stovepipes 

• NASA is now managing and controlling the requirements 

• Keep management informed, elevate issues, status reporting 

3. Establish frequent technical interchange meetings to ensure 
requirements definition and coordination 

4. Prioritize and baseline near-term requirements for “Early Science” 

5. Establish an Observatory Integration IPT to coordinate V&V planning and 
execution between the two projects, the science instrument teams, and the 
international partner 

6. Complete Early Science Observatory V&V Plan 

7. Complete long-term requirements for final SOFIA configuration (including 
ICDs, Specs, and Verification/Validation plans) (On-going) 
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Average RAC 



Requirements Deficiency 
Risk Mitigation Waterfall 



25 


• Over time, the SOFIA Requirements Deficiency Risk has been 
significantly reduced do to several mitigating actions 
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SOFIA SE Lessons Learned 




Phasing system development has 
bought time to establish a significantly 
improved set of “final” requirements 


• Valuable experience was gained accomplishing the “Early Science” 
Phase that will greatly benefit the final SOFIA system design 
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Benefits of Phasing Development 
on SOFIA Requirements 



• The “final” SOFIA system requirements will benefit from 
the knowledge gained during Early Science 

- The “near-term” requirements for meeting the “Early Science” 
goals were less stringent but still challenging 

- Several issues with requirements definitions had to be 
resolved for Early Science 

o Identified gaps and misunderstandings in requirements 

- SOFIA employed an “Agile Development” process (frequent 
iterations with collaborative feedback) to deal with these issues 

o Some degree of product rework was tolerated or procedural “work 
arounds” were employed to meet Customer expectations 

- The development team gained valuable experience 

• Phasing allowed valuable time to refine the Product 
Tree and systematically review “final” requirements 
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SOFIA SE Lessons Learned 




It takes time to become knowledgeable 
enough of complex systems to 
effectively develop “good” 
requirements 

• It took the new Program team a significant period of time to 
become proficient with the complex SOFIA systems 
- This knowledge is critical to being effective at requirement 
decomposing and establishing good traceability 
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SOFIA SE Lessons Learned 




Having a comprehensive 
specification/product tree (and ICD 
list) is critical to system integration 


• At transition, SOFIA had to deal with new “observatory-level” 
requirements that were inserted to address missing overall 
system performance values 
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SOFIA Spec Tree 
Rev B 2008 Baseline 
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Compare to May 2007 
Original Spec Tree Rev A 


Summer 2007 goal - top 4 specs approval 

SE01-003 (SOFIA), SE01-013 
(Observatory), SE01-004 (Aircraft), 
SE01-005 (MCCS) 

S Top 4 specs hove been program 
reviewed through delta SRR , MCCS 
Redesign , SOLIPT, SE&I and Program 
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Yellow - Revision in work 

Red - Not written or not approved 
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Current Rev F Spec Tree 2010 






Compare to 2008 
Baselined Spec Tree 


Between Jul 2007 & Dec 2010 : 
Program + SE&I reviewed 30 
Specification Documents, 
containing over 3000 requirements 

Forums: Delta SRRs , Design Reviews, 
IPTs, SE&I, and system assessments 


Level 6 - Subsystem /Product 
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SOFIA Interface Control Document 

Status History 



ICDs July 2007 



Approved 
Not Approved 


ICDs Dec 2010 



Approved 
Need Update 
Not Written 


49 Total ICD’s 


68 Total ICD’s 


Summer 2007 goal: 

• Identify and list all SOFIA ICDs 

• Establish initial status and 
ownership 


Between Jul 2007 & Dec 2010: 

• Several new ICDs identified 

• Several key ICDs completed 
or updated 
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Requirements Management 

Challenges 



• Availability of key personnel and conflicting priorities 

- Planners = owners = implementers = testers (all the same 
person) 

• Requirements creep due to lack of complete/baselined 
requirements or well defined interfaces 

•Traceability of design requirements completion status 
to V&V test plans/results 

- Lack of overarching program guidance and integrated test 
plans (No program integration office) 

• Although SOFIA has made a significant amount of 
progress, a lot remains to get done 

- Delta system level SRRs are on-going 

- Striving for more formality in Segment 3 (final build) 


9 February 2011 


PM Challenge 


40 





Configuration Management, Data 
Management and Related Topics 

Laura Fobel 
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SOFIA SE Lessons Learned 



To improve CM process efficiency, 
delegate CM responsibilities to the 
lowest level possible 
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The Delegation of Configuration 
Management (CM) Authority 



• SOFIA developed a hierarchy of CM Boards to drive CM 
authority down to the lowest appropriate level 

- Improves efficiency by distributing the work load 

- CM hierarchy parallels the product hierarchy 

• Over time SOFIA’S CM needs changed 

- Initially a single CM Board may have made sense on SOFIA 

- As development work expanded and system complexity grew, a 
more distributed CM process was needed 

• SOFIA established a separate Control Board to manage 
the “Observatory” configuration 

- Includes Program, Platform and Science Project members 

- Focuses on configuration management of the “integrated system” 
and related discrepancies 
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The Delegation of 
CM Authority on SOFIA 




Contract Level 
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SOFIA SE Lessons Learned 



Informal collaboration with contractors 
improves the probability of success of 
formal deliverables 


• The distributed CM process facilitated more collaboration with 
contractors 
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Informal Collaboration Improves 
Probability of Success 



Shift From Contractor Run / Government Oversight 
To Government Lead / Subcontractor Relationship 
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The Value of Informal Collaboration 



• Too much back-and-forth over the fence is inefficient 

- A significant amount of rework occurred when not enough 
informal collaboration occurred with the contractors 

• It’s important to establish a cooperative environment 
with contractors 

- SOFIA applied an “agile development” process by allowing 
informal software builds to be delivered early in the 
development process to flush-out problems prior to formal 
deliveries 

o Collaboration occurred at the lower levels and included 
stakeholders 

o Deliverables still went through the formal acceptance process to 
be baselined 
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SOFIA SE Lessons Learned 



On SOFIA it was beneficial to have a 
problem reporting process that spanned 
informal development activities and 
formal acceptance testing 
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The Value of Problem Reporting 
and Discrepancy Resolution 



• By establishing a Problem Reporting system early 
(during informal software testing) issues were identified 
and resolved sooner (prior to formal delivery) 

- Allowed customers to capture issues and collaborate with 
developers to understand and refine formal requirements 

- Supported the “Agile Development” process 

• The Observatory-level control board allowed cross- 
Project issues to be identified and resolved jointly 

- Chaired by the Program Chief Engineer 

o Provided independent authority 

- Established priorities and assignments to Projects for resolving 
integration issues 

- Facilitated communication of issues and their resolution 
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SOFIA SE Lessons Learned 



The lack of a carefully designed Data 
Management systems hinders effective 
communication and collaboration 


• SOFIA team members had a difficult time finding the 
information they needed 

-Old and obsolete data mixed with relevant data contributed to 
the problem 
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SOFIA Data Management 
Improvements 



• Established a central repository to improve control and 
management of the data originating from various 
sources 

- Reorganized the data and archived obsolete documents 

• Defined data attributes for each document 

- Product ID - Document number 

- Data retention - Export control 

- Owner - CM authority 

- Descriptive search keywords 

• Considered Configuration Management, Data 
Management, Export Control, Records Retention, and 
Data Access as part of one integrated process 
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SOFIA Sample Records 
Retention Schedule 


(SOF-1017 Category) Record Type 

NPR1441.D 
Schedule - 
Item Number 

Windchill- 

Record 

Retention 

Number 

Records (NPR 1441. ID defined or SOFIA defined) 

Records 

Location 

Retention 

Authority 






<01> Program and Project Management (PM) 





Management Plans 

Sch.8 • 101 

01-S8-101.1 

Program Plan; Project Plan; Systems Engineering Management Plan (S€MP); 
Configuration Management Plan; Manufacturing and Assembly Plans; Parameter 
Control; Electromagnetic Interference and Compatibility Control; Fracture 
ControVDamage Tolerance; Support Equipment; Education and Public Outreach; 
Continuous Improvement/Preplanned Product Improvement; Observatory 
Certification; SOFIA Observatory Integration Plan; System and Subsystem 
Integration Plans; Aircraft System Flightiest; SOFIA Observatory Ground Test; 
SOFIA Observatory Flightiest; SOFIA Science and Mission Operations; SOFIA 
Science and Mission Operations; New Technology Reporting; Training Plans; 
Software Management Plans; Software Development Plans; Safety, Reliability and 
Mission Assurance Plans; Environmental, Safety and Health Plan, Integrated 
Logistics Support Plans; Data Management Plan; Mission Statements; Operations 
Concept 

Held at office 
of record 
(ARC: N211- 
Room 320) 

101 - Permanent Record. 
3-year blocks cutoff for 
longterm programs. Can 
transfer to National 
Archives 7 years after 
cutoff. 

Agreements, Understandings and Approvals 

Sch.8-101 

01-S8-101.2 

Partnering Agreements; Memorandums of Understanding; Memorandums of 
Agreements; Program Commitments; Authorization/Approval Documents 



Schedules 

Sch.8 -101 

01-S8-101.3 

Program Milestones; Project Milestones; Schedules; Integrated Master Schedule 



Budget and Finance 

Sch.8 - 101 

01-S8-101.4 

Work Breakdown Structure and Dictionary; Budget and Cost Data; Estimates of 
budget and schedule options 



Configuration Management 

Sch.8-101 

01-S8-101.5 

Configuration Management Board (CCB) Agendas, Minutes and Review Material; 
Configuration Change Requests; Discrepancy Reports; System Test Reports; 
Waivers 



Risk Management 

Sch.8 -103 

01-S8-103.6 

Risk Management Board (RMB) Agendas. Minutes and Review Material; Risk Lists 
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Sample SOFIA Document 
Attributes 




File attributes facilitate Data 
Management, Export 
Control, Access Control 
and Records Retention 
processes. 

• Data Management 
(Document Name): SOF- 
DA-ICD-SE03-002 

• Export Control/Access 
Control: “Not Reviewed for 
Export Control” 

• Records Retention: Date 
(2003-03-28) and Records 
Retention Schedule 
reference (03-S8-103.3) 
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SOFIA CM and DM Remaining 

Challenges 



• Establishing ownership and control of all SOFIA 
documents and drawings 

- Contractors still own important information like “models” 

• Shortcomings of the Data Management System 

- Search engine and user interface complexity 

- User familiarity 

- Data access by Foreign Nationals 

• Catching up with Export Control and Records 
Retention attribute labeling 
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SOFIA SE&I Summary 


Ron Ray 
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SOFIA SE Lessons Learned 




Management must set the cultural 
tone for the importance of SE on a 
Program/Project 

• The commitment the SOFIA Management Team has made 
to SE has helped turn around the once “troubled” Program 
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SOFIA Program Documentation 
Status as of: 12/03/2010 




General Program/Project Documentation 

Science Project 

Platform Project 

SOFIA Program 

1 

Program/Project Plan 

Baseline 

Baseline 

Baseline/Rev 

2 

Systems Eng. Management Plan (SEMP) 



Rev A 

3 

Configuration Management Plan (CMP) 

Baseline 

Rev A/Updating 

Rev 3/Rev 

4 

Risk Management Plan (RMP) 



Rev B 

5 

IT Management Plan 



In Review 

6 

Data Management Plan (DMP) 

Baseline 


Baseline 

7 

Export Control Plan 



85%/Dec 

8 

Safety and Mission Assurance Plan (SMA) 

Rev B 


Baseline 

9 

System Safety Plan 


Baseline 

Baseline 

10 

Reliability & Maintainability Plan (R&M) 



In Review 

11 

Mishap Response Plan 



Rev A 

12 

Quality Assurance Plan 


Baseline 


13 

Software Development Plan 


Baseline 


14 

Software Management Plan 

Rev A 



15 

Software Assurance Plan 

Baseline 

Baseline 


16 

Platform Project IT Security Plan 


Baseline 


17 

Concept of Operations 



In Work 

19 

Work Breakdown Structure (WBS) 



Rev B 

20 

Lexicon (SOFIA glossary) 



Rev 6.6 

21 

Supplier Statement of Requirements (SSOR) 


Baseline 


22 

Integrated Master Schedule (IMS) 

New Baseline 

New Baseline 

New Baseline 

23 

Level 1 & 2 Milestones 

New Baseline 

New Baseline 

Rev E 

25 

Risk List 

Baseline 

Baseline 

Rev H 


Legend 


Complete 

Open, On-going 

New/Change Status 

Program Issue 
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SOFIA First Light Image 

May 16 th 2010 

SOFIA infrared image 
(5.4, 24, and 37 /urn) 



Visible light image 

• SOFIA is beginning to produce outstanding science 
data at a fraction of the cost of comparable space 
based observatories 


http://www.nasa.gov/mission_pages/SOFIA/ 



SOFIA First Science Image 

Dec 1 st 2010 




Visible light 
(ground-based) 


Near infrared (ESO VLT) SOFIA (FORCAST) 


• Image of the Orion star-formation region obtained by SOFIA 
compared to images obtained from ground-based telescopes 


http://www.dlr.de/DesktopDefault.aspx/tabid-1/117_read-28014/ 




Concluding Message 


* After almost being cancelled and a major Program 
structure change, the SOFIA Program operated at 
Risk with known Systems Engineering deficiencies 

* Several important strategies were employed to 
mitigate the risk 

- Established a new incremental life-cycle to complete 
system development 

- Worked more collaboratively 

- Systematically rebuilt SE&I along the way 

o Provided adequate staffing and priority 

- Made correcting requirements deficiencies a high-priority 

- Distributed CM authority 

- Tracked and status SE Progress 

* SOFIA has used Risk Management effectively to 
compensate for Systems Engineering deficiencies 
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Summary of SE Lessons Learned 



* It is never too late to fix Systems Engineering (SE) 
deficiencies 

* Breaking complex development activities into 
increments can improve the overall chance of 
success 

* Management needs clear insight on the status of 
SE products 

* SE must account for and tailor to various Center 
and cultural differences 

* “Better is the enemy of good enough” 
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Summary of SE Lessons Learned 



* Making the “Lack of Requirements Definition” a 
Program risk, is an effective way to highlight and 
address the problem 

* Phasing system development has bought time to 
establish a significantly improved set of “final” 
requirements 

* It takes time to become knowledgeable enough of 
complex systems to effectively develop “good” 
requirements 

* Having a comprehensive specification/product tree 
(and ICD list) is critical to system integration 
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Summary of SE Lessons Learned 



* To improve CM process efficiency, delegate CM 
responsibilities to the lowest level possible 

* Informal collaboration with contractors improves 
the probability of success of formal deliverables 

* On SOFIA it was beneficial to have a problem 
reporting process that spanned informal 
development activities and formal acceptance 
testing 

* The lack of a carefully designed Data Management 
systems hinders effective communication and 
collaboration 

* Management must set the cultural tone for the 
importance of SE on a Program/Project 
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